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ABSTRACT 

When gathering and presenting educational statistics, 
five principles should be considered. (1) The data must be accurate, 
valid, and complete. Limitations, weaknesses, margins of error, and 
levels of confidence should be clearly stated. (2) The data must 
include comparable information, sought in comparable ways, in 
comparable forms, from comparable sources. (3) Data that do not have 
immediate utility should be collected such that they may be useful at 
a later time. (4) Historical statistics should be collected in order 
to provide the context for interpreting long-term change, the 
significance of change, and solutions to problems. (5) Data 
collection procedures should be simple and methods of reporting 
should be easy to use. The comparability of data concerning 
educational quality must be improved. The United States Department of 
Education and the Council of Chief State School Officers should 
cooperate with state officials to standardize data collection and 
presentation. Further research is needed in the areas of teachers' 
qualifications, student dropouts, art and music instruction, and 
programs to publish research results. (GDC) 
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TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) " 

It is easy enough, as many have shown, to find serious 
deficiencies in the gathering and presentation of educational 
statistics. Yet in reflecting on flaws in existing data 
and on the consequences of their weakness, we often lose 
sight of the general principles that should guide any 
improvements and revisions of them. As a prelude to some 
specific concerns and at the risk of reviewing some hackneyed 
themes, I should like to consider these principles, of which 
in my view there are five. 

The first is that of accuracy. The raw data used to 
construct statistical series must be valid and complete; 
their weaknesses and limitations must be specified; and 
probable margins of error and levels of confidence must 
be stated. Only in this way can faith in the strength of 
the statistics be raised, their utility increased, and 
ill-conceived employment of them — for ideological, 
propaganda, and other ends — be curbed. 

Second to accuracy in order but not in importance is 
the principle of comparability. Accurate data to not assure 
their utility unless comparable information has been sought 
in comparable ways in comparable forms from comparable 
sources. The failure to create comparable information 
subverts all efforts accurately to assess differences and 
changes in educational circumstances between places and over 
time and thereby weakens or nullifies policies built upon 
them. 

The third principle is that of potential utility. 
Educational data should be jollected even in the absence of 
contemporary or immediate needs in policy-making, and they 
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should be collected in such form that they can be re- 
formulated for changing uses. Failure to collect data that 
may seem superfluous for immediate presentation can reduce 
the effectiveness of future policies by inhibiting their 
substantiation and the evaluation of their historical 
significance. 

Related to potential utility is a fourth principle, 
that of history. Too often ignored but essential to the 
evaluation of the meaning of any changes over time, historical 
statistics provide the context for the interpretation of 
long-term change. Without historical statistics, all 
claims about trends, the significance of data, and the 
solutions to problems are suspect. 

Fifth and final is the modest, but essential, principle 
of ease. if responses to requests for data become burdensome, 
or if data, once collected, cannot be easily used or 
assessed, statistics decline in appeal. Therefore, data 
reporting must be made simple for those from whom it is 
solicited, and data samples should be collected in place of 
statistics on a complete universe whenever possible. 
Relatedly, data that lend themselves readily to scholarly 
analysis — even to such basic and simple uses as 
correlations — should be collected so that the function of 
NCES can remain primarily that of data collection rather 
than analysis, which should be carried out by others. 
Furthermore, data should be presented in such a way that 
local reporting units, be they individual schools or entire 
school districts, can assess their relative positions in 
comparison with averages for similar reporting units. 

Taken together, these principles, if applied, will 
help ensure the integrity and applicability of the statistical 
information upon which we must in part base all general 
educational policies. Yet they will not at all times be of 
equal moment. Occasionally, circumstances may require 
emphasis upon a single one in order to increase confidence 
in the strength of all data. In my view, now is such a 
time. What requires immediate attention, in order to secure 
the utility of what information, however accurate, we now 
have, is the comparability of educational data. And the 
Department of Education must take the lead in this effort. 

For many reasons, the American public now seeks to be 
assured of improvements in education at all levels, especially 
in the primary and secondary schools. And, characteristically, 



it wants information that compares present conditions with 
those of the recent past and conditions in one jurisdiction 
with those in others. Yet the plain fact of the matter is 
that the data available to provide such comparisons are 
embarrassingly weak. The public is being misled by their 
use. And policies based upon them may therefore be without 
foundation or desired effect. For these reasons, even 
before attempts are made to improve the collection of 
accurate raw data, the data that are solicited must be 
collected in ways that allow accurate comparisons of 
educational conditions among jurisdictions and across time. 
This is not to say that the search for improved accuracy 
should not proceed. Rather, the public as well as policy- 
makers must be assured that, even in the absence of reliable 
data, the nature, direction, and extent of change in American 
education is accurately represented by the data that we do 
have. That cannot be done now. 

Moreover, the data that are gathered and published must 
be consistent over time. Too often, the existing data series 
are presented differently, due both to altered data-gathering 
methods and changed survey questions, from one year to the 
next. This renders virtually nugatory all attempts to 
evaluate changes in educational conditions. 

How should the endeavor to improve the comparability of 
data be undertaken? Despite the sensitivity, as misplaced 
as it often is, toward the centralization of educational 
policy, the attempt to collect comparable data should be 
led by a national agency or organization, if not by the 
Department of Education then under the auspices of the 
Council of Chief State School Officers. In any case, both 
must act in concert with state officials to standardize 
data collection and presentation. 

One problem facing the Department and NCES is the use 
of proxies for collecting data, as a result of which the 
government's need for information depends upon the needs of 
other organizations and agencies and often falls victim to 
them. This is most visible in data from sample surveys, 
often undertaken at a third remove — that is, at the behest 
of clients of reporting organizations. If adequate and 
consistent proxy surveys and data cannot be secured, then 
the Department must try tc secure them directly or by 
contract. 

Additions to, and changes in, existing data series will 
not be of much use in my estimation without speedy attention 



to these general and encompassing obstacles that now stand 
in the way of sound data collection. Nevertheless, some 
more specific matters also require tending to if the 
Department's data series are to be made more useful. 
Omissions in the data, especially those concerning the 
quality of primary and secondary schooling, must be filled. 
Although all efforts to measure the quality of education 
must end in approximations, they must nonetheless be made 
and the glaring lacunae in existing data closed. 

For instance, despite the intensive debate regarding 
the preparation and qualifications of school teachers, we 
have no adequate current or long-term information to that 
effect. Data series, by state, about teachers' levels of 
education (bachelor, master, and doctoral degrees and 
credit hours beyond certification), permanent and temporary 
certification, the ratio of those teaching with and without 
certification in their field of instruction, the percentage 
of those who have switched teaching fields, the percentage 
of those teaching subjects in which they were not initially 
trained, student-teacher ratios by fields, the results of 
newly-instituted competency tests, and similar matters 
should be maintained and presented historically. Similarly, 
we should have measures of scores on the College Board 
and other advanced placement and achievement tests, broken 
out by states and public and private schools. Efforts to 
measure changes in college entrance requirements should be 
endeavored too, as well as changes in the relative proportion 
of advanced and AP courses and general courses and in the 
number and proportion of students taking each. 

In many cases, data are unwisely aggregated or 
aggregated in forms that reduce their usefulness. Despite 
jurisdictional realities, a better unit for comparison of 
much data is probably the standard metropolitan area rather 
than the state, at least for urban schools* Analogously, 
data regarding public and private schools should be dis- 
tinguished. Much of the data fail to reflect the dual 
system of American education at all levels; and even those 
few efforts to distinguish between different kinds of 
private and religious schools are not carried out con- 
sistently in the series now published. 

Discontinuities in students' school experiences are 
not adequately assessed either. Measures of dropping out, 
notoriously weak as they may be, fail to distinguish 



between chosen interruptions in schooling ("stopping out") 
and involuntary interruptions (such as moving from one 
school to another). These data could be collected via 
sample surveys and with the assistance of the Bureau of 
the Census. In fact, one measure of the stability of schools 
might be a simple ratio of new to returning students per 
jurisdiction. 

Of the curricular information that should be added 
to existing data series, that concerning the state of art 
and music should have high priority. The omission of these 
subjects from consideration by authors of the recent flood 
of national reports on schooling was egregious. By contrast, 
already collected data on so-called remedial courses are 
probably the weakest that are published, given the wide 
differences in definitions of remediatioi< by school districts 
and individual, especially private, schools. Such data 
could be omitted. 

Finally, some attention should be given to publishing 
revised data as better information is collected or errors 
in previously published data are discovered — as, for 
e .ample, the Treasury Department routinely does with its 
periodic economic statistics. For example, it is exceedingly 
unlikely that the percentage of 18-19 year olds graduating 
from high school rose by 5.4 per cent between 1975 and 1976 
and then dropped 3.3 per cent the next year. Chances are 
that the data for 1976 are suspect. Such evident abnornalitie 
should be carefully checked; and if the data are found to be 
erroneous, they should be revised in all subsequent 
publications. 

In all of these modest ways, and in others that will 
be proposed, the foundations of knowledge about American 
education will be greatly strengthened. 
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